Master File¶
This notebook runs the GP backtest, NN-IV training and backtesting, SSVI backtesting and trinomial tree backtesting. Finally all results are compared.
Help on Importing files¶
If you run this notebook on google colab you need to upload python scripts on the left panel. To that end click on the left "Files" (or "Fichiers" in French) and drag and drop :
- python scripts from the "code" folder of github repository.
- csv files or xls files from "data" folder.
- Tensorflow model (files with .data or .index extensions) in the "Results" folder if you want to use the neural network while avoiding the training step.
- csv files in "Results" folder if you want to load Results presented in our paper.
In [2]:
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
from IPython.display import display
import sklearn as skl
import sys
formerPath = sys.path
sys.path.append('./code/')
sys.path.append('./BS/')
import os
formerStdOut = sys.stdout
import bootstrapping
import dataSetConstruction
import backtest
import BS
import loadData
import plotTools
import SSVI
import SSVIUnconstrained
import neuralNetwork
import experiments
import importlib
plt.rcParams['figure.dpi'] = 50
In [3]:
sys.stdout = formerStdOut
Load data¶
In order to reproduce our paper experiments, execute cells from part "Load preformatted data".
Each source of data produces the following objects :
- bootstrap manages discounting and dividend.
- dataSet contains the training set.
- dataSetTest contains the testing set.
- $S0$ the spot value of the underlying
Load preformatted data ("Reload")¶
You should execute that cell, if you want to keep the same training set and testing set as those used in the paper.
File required :
- testingDataSet.csv
- trainingDataSet.csv
- dfCurve.csv
DAX Data ("DAX")¶
For each day you need to load six csv files :
- underlying.csv for the stock value.
- locvol.csv for the local volatility calibrated with tree pricing and tikhonov volatility (see Crépey (2002)).
- dividend.csv for dividend extracted from put-call parity.
- discount.csv for zero-coupon curve.
- dataTrain.csv for prices and/or implied volatility used in training set.
- dataTest.csv for prices and/or implied volatility used in testing set.
- "8_8_2001__filterdax.dat" for tikohnov local volatility.
From DAX data from .dat files ("DAX dat files")¶
For a day x you must load :
- "x.dat.modelparam.dat"
- "x.dat.impliedvol.dat"
- "x.dat.calibr.out.dat"
- "x.dat"
Eurostoxx data ("Eurostoxx")¶
File required : Data_EuroStoxx50_20190110_all_for_Marc.xlsx
Formatting data¶
Boostsrapping Rate Curve¶
We assume a piecewise constant discount short rate $r$ and a piecewise constant dividend short rate $q$.
We estimate the "zero coupon dividend" $D(T) = e^{-\int_{0}^{T} q_s ds}$ by regressing it against maturity : $$e^{-\int_{0}^{T} q_s ds} = \frac{C(T,K) - P(T,K) + K e^{-\int_{0}^{T} r_s ds}}{S_0}$$
Then we have $\hat{q}_t = - \frac{ \log{D(\overline{T})} - \log{D(\underline{T})} }{ \overline{T} - \underline{T} }$ with $\overline{T}$ the smallest discretized maturity greater than $T$ and $\underline{T}$ the grestest discretized maturity inferior than $T$.
bootstrap object has several members :
- riskFreeIntegral corresponds to $I_T = \int_{0}^{T} r_u du$.
- riskFreeSpline corresponds to $r_u$ evaluated on a subgrid. Interpolated as step function from zero coupons
- divSpreadIntegral corresponds to $I_T = \int_{0}^{T} q_u du$, can be negative.
- divSpline corresponds to $q_u$ evaluated on a subgrid, can be negative.
These curve should satisfy the call-put parity.
Change of variable¶
Neural network on modified prices with modified strike as input such that discounting and dividend don't intervene in Dupire formula calculation.
- In presence of dividend rate $d$ and risk free rate $r$ Dupire formula is : $$\sigma^2(T,K) = 2 \frac{ \partial_T P(T,K) + (r-q) K \partial_K P(T,K) + qP(T,K)}{K² \partial_{K}^2 P(T,K)}$$ with Strike $K$, Maturity $T$, dividend rate $q$ and risk-free rate $r$, $P$ our pricing function.
- We apply the following change of variable : $$ w(T,k) = \exp{(\int_{0}^{T} q_t dt)} P(T,K)$$ with $K = k \exp{(\int_{0}^{T} (r_t - q_t) dt)} $.
- Then Dupire equation becomes : $\sigma^2(T,K) = 2 \frac{ \partial_T w(T,k)}{k² \partial_{k}^2 w(T,k)}$.
- If we learn the mapping $v$ with a neural network then we obtain quickly by adjoint differentiation $\partial_T w$ and $\partial_{k²}^2 w$ and therefore $\sigma$.
- $k$ corresponds to "ChangedStrike" column.
- $\exp{(\int_{0}^{T} q_t dt)}$ corresponds to "DividendFactor" column.
In [4]:
data = experiments.loadAndFormatData("Reload")
dataSet, dataSetTest, S0, bootstrap, KMin, KMax, midS0, scaler, scaledDataSet, scaledDataSetTest, volLocaleGridDf = data
Neural Network : Learning Implied volatility¶
Yon can skip the training step by loading in the left panel in colab workspace tensorflow models. These models are contained in the results folder of github repository.
In [43]:
hyperparameters = {}
#penalization coefficient
hyperparameters["lambdaLocVol"] = 0.0
hyperparameters["lambdaSoft"] = 0.0
hyperparameters["lambdaGamma"] = 0.0
#Derivative soft constraints parameters
hyperparameters["lowerBoundTheta"] = 0.0001
hyperparameters["lowerBoundGamma"] = 0.0
#Local variance parameters
hyperparameters["DupireVarCap"] = 10.0
hyperparameters["DupireVolLowerBound"] = 0.03
hyperparameters["DupireVolUpperBound"] = 0.70
#Learning scheduler coefficient
hyperparameters["LearningRateStart"] = 0.01
hyperparameters["Patience"] = 200
hyperparameters["batchSize"] = 50
hyperparameters["FinalLearningRate"] = 1e-6
hyperparameters["FixedLearningRate"] = False
#Training parameters
hyperparameters["nbUnits"] = 100 #number of units for hidden layers
hyperparameters["maxEpoch"] = 10000#10000 #maximum number of epochs
hyperparameters["UseLogMaturity"] = True
hyperparameters["nbEpochFork"] = 0
hyperparameters["lambdaFork"] = 0.0
hyperparameters["HolderExponent"] = 4.0
In [ ]:
In [ ]:
In [5]:
res = experiments.trainNeuralNetworkImpliedVolatility(dataSet,
hyperparameters,
scaler,
False)
y_pred4G, volLocale4G, dNN_T4G, gNN_K4G, lossSerie4G = res
In [44]:
resTrain, resTest = experiments.evaluateNeuralNetworkArbitrableImpliedVolatility(dataSet,
dataSetTest,
hyperparameters,
scaler,
KMin,
KMax,
S0, midS0,
bootstrap)
In [ ]:
With arbitrage constraint¶
In [4]:
hyperparameters = {}
#penalization coefficient
hyperparameters["lambdaLocVol"] = 1.0
hyperparameters["lambdaSoft"] = 100.0
hyperparameters["lambdaGamma"] = 10000.0
#Derivative soft constraints parameters
hyperparameters["lowerBoundTheta"] = 0.000001
hyperparameters["lowerBoundGamma"] = 0.0
#Local variance parameters
hyperparameters["DupireVarCap"] = 10.0
hyperparameters["DupireVolLowerBound"] = 0.03
hyperparameters["DupireVolUpperBound"] = 1.00
#Learning scheduler coefficient
hyperparameters["LearningRateStart"] = 0.01
hyperparameters["Patience"] = 200
hyperparameters["batchSize"] = 50
hyperparameters["FinalLearningRate"] = 1e-6
hyperparameters["FixedLearningRate"] = False
#Training parameters
hyperparameters["nbUnits"] = 100 #number of units for hidden layers
hyperparameters["maxEpoch"] = 10000#10000 #maximum number of epochs
hyperparameters["UseLogMaturity"] = True
hyperparameters["nbEpochFork"] = 0
hyperparameters["lambdaFork"] = 0.0
hyperparameters["HolderExponent"] = 2.0
In [ ]:
In [5]:
res = experiments.trainNeuralNetworkImpliedVolatility(dataSet,
hyperparameters,
scaler,
True)
y_pred4G, volLocale4G, dNN_T4G, gNN_K4G, lossSerie4G = res
In [6]:
resTrain, resTest = experiments.evaluateNeuralNetworkArbitrableFreeImpliedVolatility(dataSet,
dataSetTest,
hyperparameters,
scaler,
KMin,
KMax,
S0, midS0,
bootstrap)
Monte Carlo and PDE repricing backtests¶
During Monte Carlo backtest, each option in testing set is priced with an underlying which is diffused with the following SDE : $$ d\log{S_t} = \left( r_t - q_t - \frac{\sigma_{NN}^2(t, S_t)}{2} \right) dt + \sigma_{NN}(t, S_t) dW_t$$ with $\sigma_{NN}$ the neural local volatility function.
Due to computation time issue we avoid to make millions of call to neural network and we interpolate linearly neural local volatility obtained on one the two possible grid :
- the testing grid i.e. nodes $(T,K)$ of the testing set.
- an artificial grid of 10000 points to check local volatility is correctly interpolated/extrapolated. That grid is the smallest rectangle containing the minimum and maximum maturities and the minimum and maximum strikes of our dataset (union of testing and training set).
During PDE backtest, we used a crank-nicholson scheme to revaluate each option in our testing set. Time step corresponds to one day and space grid has 100 points.
In [7]:
# Function which evaluates neural local volatility when neural network is fitted on implied volatilities
def neuralVolLocaleGatheral(s,t):
vLoc = neuralNetwork.evalVolLocaleGatheral(neuralNetwork.NNArchitectureVanillaSoftGatheral,
s, t,
dataSet,
hyperparameters,
scaler,
bootstrap,
S0,
modelName = "convexSoftGatheralVolModel")
return vLoc.dropna()
In [8]:
nbTimeStep = 100
nbPaths = 10000
In [9]:
resBacktest = experiments.backTestLocalVolatility(neuralVolLocaleGatheral,
volLocaleGridDf,
dataSetTest,
nbTimeStep,
nbPaths,
KMin,
KMax,
S0,
bootstrap,
"NeuralImpliedVolatility")
volLocalGridRefinedG, volLocalGridTestG, mcResVolLocaleRefinedG, mcResVolLocaleTestG, pdeResVolLocaleRefinedG, pdeResVolLocaleTestG = resBacktest
In [ ]:
Neural network : Learning Price¶
Yon can skip the training step by loading in the left panel in colab workspace tensorflow models. These models are contained in the results folder of github repository.
In [45]:
hyperparameters = {}
#penalization coefficient
hyperparameters["lambdaLocVol"] = 0.0#1#0.001#0.01 #100
hyperparameters["lambdaSoft"] = 0.0#100#0.0001#10#10 #100
hyperparameters["lambdaGamma"] = 0.0#10000#100#10 #10000
#Derivative soft constraints parameters
hyperparameters["lowerBoundTheta"] = 0.000001#0.01
hyperparameters["lowerBoundGamma"] = 0.0000001
#Local variance parameters
hyperparameters["DupireVarCap"] = 10.0
hyperparameters["DupireVolLowerBound"] = 0.03
hyperparameters["DupireVolUpperBound"] = 1.00
#Learning scheduler coefficient
hyperparameters["LearningRateStart"] = 0.01
hyperparameters["Patience"] = 10000
hyperparameters["batchSize"] = 50
hyperparameters["FinalLearningRate"] = 1e-6
hyperparameters["FixedLearningRate"] = False
#Training parameters
hyperparameters["nbUnits"] = 100 #number of units for hidden layers
hyperparameters["maxEpoch"] = 10000#10000 #maximum number of epochs
hyperparameters["UseLogMaturity"] = False
hyperparameters["nbEpochFork"] = 0#10000
hyperparameters["lambdaFork"] = 0.0#1000.0
hyperparameters["HolderExponent"] = 2.0
In [ ]:
In [15]:
res = experiments.trainNeuralNetworkPrice(dataSet,
hyperparameters,
scaler,
False)
y_pred4, volLocale4, dNN_T4, gNN_K4, lossSerie4 = res
In [47]:
resTrain, resTest = experiments.evaluateNeuralNetworkArbitrablePrice(dataSet,
dataSetTest,
hyperparameters,
scaler,
KMin,
KMax,
S0, midS0,
bootstrap)
In [ ]:
with arbitrage constraints.¶
In [17]:
hyperparameters = {}
#penalization coefficient
hyperparameters["lambdaLocVol"] = 5.0 * 1e0#1#0.001#0.01 #100
hyperparameters["lambdaSoft"] = 1e4#100#0.0001#10#10 #100
hyperparameters["lambdaGamma"] = 1e7#10000#100#10 #10000
#Derivative soft constraints parameters
hyperparameters["lowerBoundTheta"] = 0.000001#0.01
hyperparameters["lowerBoundGamma"] = 0.0000001
#Local variance parameters
hyperparameters["DupireVarCap"] = 10.0
hyperparameters["DupireVolLowerBound"] = 0.03
hyperparameters["DupireVolUpperBound"] = 1.00
#Learning scheduler coefficient
hyperparameters["LearningRateStart"] = 0.01
hyperparameters["Patience"] = 10000
hyperparameters["batchSize"] = 50
hyperparameters["FinalLearningRate"] = 1e-6
hyperparameters["FixedLearningRate"] = False
#Training parameters
hyperparameters["nbUnits"] = 100 #number of units for hidden layers
hyperparameters["maxEpoch"] = 10000#10000 #maximum number of epochs
hyperparameters["UseLogMaturity"] = False
hyperparameters["nbEpochFork"] = 0#10000
hyperparameters["lambdaFork"] = 0.0#1000.0
hyperparameters["HolderExponent"] = 2.0
In [ ]:
In [19]:
res = experiments.trainNeuralNetworkPrice(dataSet,
hyperparameters,
scaler,
True)
y_pred4, volLocale4, dNN_T4, gNN_K4, lossSerie4 = res
In [20]:
resTrain, resTest = experiments.evaluateNeuralNetworkArbitrableFreePrice(dataSet,
dataSetTest,
hyperparameters,
scaler,
KMin,
KMax,
S0, midS0,
bootstrap)
Monte Carlo and PDE repricing backtests¶
In [21]:
# Function which evaluates neural local volatility when neural network is fitted on prices
def neuralVolLocalePrix(s,t):
vLoc = neuralNetwork.evalVolLocale(neuralNetwork.NNArchitectureVanillaSoftDupire,
s, t,
dataSet,
hyperparameters,
scaler,
bootstrap,
S0,
modelName = "convexSoftVolModel")
return vLoc.dropna()
In [22]:
nbTimeStep = 100
nbPaths = 10000
In [23]:
resBacktest = experiments.backTestLocalVolatility(neuralVolLocalePrix,
volLocaleGridDf,
dataSetTest,
nbTimeStep,
nbPaths,
KMin,
KMax,
S0,
bootstrap,
"NeuralPrice")
volLocalGridRefined, volLocalGridTest, mcResVolLocaleRefined, mcResVolLocaleTest, pdeResVolLocaleRefined, pdeResVolLocaleTest = resBacktest
SSVI Unconstrained¶
Run SSVI Model¶
Implementation is inspired from Tahar Ferhati code :
- Ferhati, T. (2020). Robust Calibration For SVI Model Arbitrage Free. Available at SSRN 3543766.
Black crosses in local volatility plots locate arbitrage violations thanks to finite difference sensitivities.
In [24]:
ssviModel = experiments.trainSSVIModel(dataSet, S0, bootstrap, False)
In [25]:
serie, dT, hk, dK, locVolSSVI, density = experiments.evalSSVIModel(dataSet,
ssviModel,
KMin, KMax,
S0,
bootstrap,
"SSVIUnconstrainedTraining")
In [26]:
serieTest, dTTest, hkTest, dKTest, locVolSSVITest, densityTest = experiments.evalSSVIModel(dataSetTest,
ssviModel,
KMin, KMax,
S0,
bootstrap,
"SSVIUnconstrainedTesting")
In [ ]:
SSVI Constrained¶
Run SSVI Model¶
Implementation is inspired from Matlab code of Philipp Rindler :
- Philipp Rindler (2020). Gatherals and Jacquier's Arbitrage-Free SVI Volatility Surfaces (https://www.mathworks.com/matlabcentral/fileexchange/49962-gatherals-and-jacquier-s-arbitrage-free-svi-volatility-surfaces), MATLAB Central File Exchange. Retrieved November 20, 2020.
In [16]:
ssviModel = experiments.trainSSVIModel(dataSet, S0, bootstrap, True)
In [17]:
serie, dT, hk, dK, locVolSSVI, density = experiments.evalSSVIModel(dataSet,
ssviModel,
KMin, KMax,
S0,
bootstrap,
"SSVIConstrainedTraining")
In [18]:
serieTest, dTTest, hkTest, dKTest, locVolSSVITest, densityTest = experiments.evalSSVIModel(dataSetTest,
ssviModel,
KMin, KMax,
S0,
bootstrap,
"SSVIConstrainedTesting")
Backtest Local volatility SSVI¶
In [19]:
def neuralVolLocaleSSVI(s,t):
volLocaleGrid = pd.DataFrame(np.vstack((np.ravel(s), np.ravel(t))).T,
columns = ["Strike", "Maturity"]).set_index(["Strike","Maturity"],drop=False)
volLocaleGrid["ChangedStrike"] = bootstrap.changeOfVariable(volLocaleGrid["Strike"], volLocaleGrid["Maturity"])[0]
volLocaleGrid["logMoneyness"] = np.log(volLocaleGrid["ChangedStrike"] / S0)
volLocaleGrid["OptionType"] = np.ones_like(volLocaleGrid["logMoneyness"])
_, _, _, locVolSSVI, _ = SSVI.finiteDifferenceSVI(volLocaleGrid, ssviModel.eval)
return locVolSSVI.dropna()
In [31]:
nbTimeStep = 100
nbPaths = 10000
In [32]:
resBacktestSSVI = experiments.backTestLocalVolatility(neuralVolLocaleSSVI,
volLocaleGridDf,
dataSetTest,
nbTimeStep,
nbPaths,
KMin,
KMax,
S0,
bootstrap,
"SSVI")
volLocalGridRefinedSSVI, volLocalGridTestSSVI, mcResVolLocaleRefinedSSVI, mcResVolLocaleTestSSVI, pdeResVolLocaleRefinedSSVI, pdeResVolLocaleTestSSVI = resBacktestSSVI
GP Backtesting¶
The purpose of this section is to load the GP local volatility surface and perform the Monte-Carlo backtest of the option prices thanks to the GP local volatility surface. Note that the GP local volatility is generated by running the Matlab code in the "code/GP" folder. See Section "GP Local Volatility Backtests" below for further details of the backtests.
Load GP Results¶
This section loads result from the Matlab experiments. See code/GP folder to access the matlab script.
In [20]:
nnGP = experiments.loadGPResults(dataSet, dataSetTest, S0, bootstrap, KMin, KMax, volLocaleGridDf)
GP Local Volatility Backtests¶
In [6]:
nbTimeStep = 100
nbPaths = 10000
In [35]:
resBacktestGP = experiments.backTestLocalVolatility(nnGP,
volLocaleGridDf,
dataSetTest,
nbTimeStep,
nbPaths,
KMin,
KMax,
S0,
bootstrap,
"GP")
volLocalGridRefinedGP, volLocalGridTestGP, mcResVolLocaleRefinedGP, mcResVolLocaleTestGP, pdeResVolLocaleRefinedGP, pdeResVolLocaleTestGP = resBacktestGP
Results Comparison¶
In [11]:
experiments.compareResults(dataSet, dataSetTest, S0, bootstrap)
In [11]:
#Sanity check for backtest with a flat local volatility surface equal to 0.2.
experiments.backTestUnitTest(volLocaleGridDf,
dataSetTest,
nbTimeStep,
nbPaths,
KMin,
KMax,
S0,
bootstrap,
0.2)
In [ ]:
In [ ]: